fix(migrations): add connector settings to existing databases - #13

Problem

The v0.2.0 database is already at migration 1, but does not contain connector_settings. That table was later added directly to migration 0001. Upgrading an existing database therefore skips its creation: the server starts and /healthz returns 200, while the authenticated connector catalog returns 500. A fresh database works.

This blocks releasing the fixes from #11 and #12 to installations using the historical schema. Refs felinics/Memoh#1168; the original reporter's database has not been inspected.

Change

Add migration 0002 to create connector_settings if it is missing. Keep 0001 unchanged and preserve existing settings, tokens, connector configuration, and connections. The 0002 down migration retains the table because current version-1 databases already own it; 0001 down still performs complete teardown.

Add a byte-identical v0.2.0 SQL fixture and PostgreSQL regressions for legacy upgrades, current version-1 databases, repeated and concurrent startup, schema isolation, and 2 → 1 → 2 transitions.

Document in docs/development.md that existing databases upgrade in place, that a migration is frozen once merged (every push to main publishes latest), and how to roll back. AGENTS.md now says never to edit a merged migration, the mistake that left connector_settings out of upgraded databases.

Validation

• Regression failed before adding 0002, then passed.
• TEST_DATABASE_URL=<isolated PostgreSQL> GOFLAGS=-count=1 mise run test: all Go modules passed.
• In packages/service, go test -race . -count=1 -timeout 60s and scoped go vet . passed.
• Independent review reran go test -race -run '^TestMigrate' -count=5 -timeout=90s -v .: 45 top-level passes, no skips or race reports.
• Docker acceptance using an independently cloned, clean build (vcs.revision=d87de04…, vcs.modified=false, image binary SHA-256 verified) of the committed API binary and PostgreSQL 18: historical v0.2.0, current version 1 with GitHub disabled, and fresh database all passed first startup and restart. Bearer-authenticated /v1/connectors and cookie-authenticated /admin/connectors returned 200; unauthenticated catalog access returned 401.
• Existing tokens worked without bootstrap reseeding, seeded configuration/connection/token rows remained unchanged, and disabled settings retained their original timestamp. Migration metadata was clean at version 2.
• Rollback acceptance on PostgreSQL 18 with the official v0.2.0 and v0.3.0 images and the PR image (search_path=connect_it, plus a host-owned public.schema_migrations at version 2): swapping only the image exits with no migration found for version 2. Both the documented SQL in a plain psql session and migrate/migrate:v4.19.1 … goto 1 returned the service schema to 1|f without touching the host table. The old image then served /healthz and bearer /v1/connectors with 200 (401 unauthenticated), and upgrading again returned to 2|f with connector_settings unchanged.
• Human QA and production upgrade.

The Docker test used synthetic credentials in a disposable database. The local running GitHub integration remains on the official image from #12; no production database was changed.

Rollback and release

Older binaries only embed migration 0001 and refuse to start at database version 2, so changing only the image is insufficient. The binary only migrates up: stop every instance, then revert the service's schema by hand. 0002's down step keeps the table and its settings, so 2 → 1 is one statement (replace connect_it with the service's schema; expect UPDATE 1):

UPDATE connect_it.schema_migrations SET version = 1 WHERE version = 2 AND NOT dirty;

The table name must be schema-qualified: in a database shared with a host application, a plain psql session resolves schema_migrations to the host's table. docs/development.md also documents the general migrate … goto <version> path. After rolling back to v0.1.1 or v0.2.0, which ignore connector_settings, disabled connectors are available again until the next upgrade.

Release v0.3.1 after this change merges, then record the released source SHA and image digest for downstream Memoh and Memoh-Cloud upgrades.